Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.x] Remove square brackets from ipv6 host on PostgreSQL and refactor changes from PR 310 #316

Conversation

richard67
Copy link
Contributor

@richard67 richard67 commented Oct 27, 2024

Pull Request for Issue #

Alternative to #315 .

Summary of Changes

This pull request (PR) adds an option to the setHostPortSocket method, which was added with PR #310 , to control if the square brackets around an IP v6 address in the host parameter should be kept ("MySQLi" and "MySQL (PDO)") or if they should be removed ("PostgreSQL (PDO)") in the connection parameters.

The default is true to keep the square brackets. For "PosgreSQL (PDO)" it is set to false so the quare brackets are removed.

This makes it possible to specify an IP v6 address together with a port number for "PostgreSQL (PDO)" in the same way as it already works for "MySQLi" and "MySQL (PDO)".

See also PR #315 for details.

In addition, this PR changes the setHostPortSocket method to not implicitly modify $this->options but to use function parameters and return value, and it renames that method to extractHostPortSocket to reflect that change.

The change to use function parameters and a return value makes it easier to provide unit tests for that method.

In the PDO driver, the original options are not modified but local variables are used. This makes sure that the options are not modified multiple times with each re-connect, which would cause problems with the removal of square brackets because the colons in an IP v6 address would cause wrong port number to be detected at the 2nd run.

In the MySQL driver this PR keeps it like it was so the behaviour is like before.

Finally, this PR adds a unit test for the extractHostPortSocket method.

Request for comments - RFC

As the extractHostPortSocket method is protected, the unit test uses reflection to make it accessible.

An alternative would be to change that method to public.

@alikon @Hackwar @HLeithner @muhme and other reviewers: What do you think? Leave the unit test as it is? Or make the method public and test it without reflection?

Testing Instructions

See CMS issue joomla/joomla-cms#43902 and joomla-projects/joomla-cypress#36 ,

And check in unit test logs in Drone if they contain a successful test "Pure host name or IP address and port or socket can be extracted from the host name option".

Documentation Changes Required

Don't know.

@richard67 richard67 changed the title [3.x] Remove square brackets from ipv6 host on PostgreSQL [3.x] Remove square brackets from ipv6 host on PostgreSQL and refactor changes from PR 310 Oct 28, 2024
@richard67 richard67 marked this pull request as ready for review October 28, 2024 13:31
@richard67
Copy link
Contributor Author

@alikon @muhme Could you test this one here? Just check if the changes from #310 are still working and if IP v6 addresses enclosed into square brackets are working. That would be much appreciated.

@richard67
Copy link
Contributor Author

@alikon @Hackwar @HLeithner @muhme and possible other other reviewers: Please check the "Request for comments - RFC" section in the description and let me know your opinion. Thanks in advance.

@muhme
Copy link

muhme commented Oct 29, 2024

Running System Tests with current 5.2-dev and JBT version 1.1.15 and bash command line:

  1. Create 5.2-dev IPv6 installation with all needed patches
# joomla-cms-43968 and joomla-cms-44084 are already merged in 5.2
scripts/create 52 IPv6 database-310 database-316 joomla-cypress-33 joomla-cypress-36
  1. Preparing tests with non-default database ports by setting up three port forwardings in the Cypress container:
docker exec jbt_cypress bash -c 'apt-get update && apt-get upgrade -y && apt-get install socat -y'
docker exec -d jbt_cypress socat 'TCP6-LISTEN:4711,fork,reuseaddr' 'TCP6:[fd00::11]:3306'
docker exec -d jbt_cypress socat 'TCP6-LISTEN:4712,fork,reuseaddr' 'TCP6:[fd00::12]:3306'
docker exec -d jbt_cypress socat 'TCP6-LISTEN:4713,fork,reuseaddr' 'TCP6:[fd00::13]:5432'
  1. ✅ Test IPv6 with non-default port number using test.sh as one-liner script with Cypress configuration, the Web Installer installation step, and one test specification that uses custom database commands.

  2. ✅ Test IPv6 with unset port number by adapting test.sh and running it again:

types=("MySQLi" "MySQL (PDO)" "MySQLi" "MySQL (PDO)" "PostgreSQL (PDO)")
hosts=("fd00::11" "fd00::11" "fd00::12" "fd00::12" "fd00::13")
ports=("" "" "" "" "")
  1. ✅ Test IPv4 with unset port number by adapting test.sh and running it again:
types=("MySQLi" "MySQL (PDO)" "MySQLi" "MySQL (PDO)" "PostgreSQL (PDO)")
hosts=("10.0.0.11" "10.0.0.11" "10.0.0.12" "10.0.0.12" "10.0.0.13")
ports=("" "" "" "" "")
  1. ✅ Test IPv4 with non-default port number by adapting test.sh and running it again:
types=("MySQLi" "MySQL (PDO)" "MySQLi" "MySQL (PDO)" "PostgreSQL (PDO)")
hosts=("10.0.0.7" "10.0.0.7" "10.0.0.7" "10.0.0.7" "10.0.0.7")
ports=("4711" "4711" "4712" "4712" "4713")
  1. ✅ Test hostname with unset port number by adapting test.sh and running it again:
types=("MySQLi" "MySQL (PDO)" "MySQLi" "MySQL (PDO)" "PostgreSQL (PDO)")
hosts=("jbt_mysql" "jbt_mysql" "jbt_madb" "jbt_madb" "jbt_pg")
ports=("" "" "" "" "")
  1. ✅ Test hostname with non-default port number by adapting test.sh and running it again:
types=("MySQLi" "MySQL (PDO)" "MySQLi" "MySQL (PDO)" "PostgreSQL (PDO)")
# Using docker container name and not 'jbt_cypress' as it’s an invalid hostname (will be fixed in the next JBT version).
hosts=("77b3b08bd2ff" "77b3b08bd2ff" "77b3b08bd2ff" "77b3b08bd2ff" "77b3b08bd2ff")
ports=("4711" "4711" "4712" "4712" "4713")
  1. 🟥 Test Unix sockets by adapting test.sh and running it again:
types=("MySQLi" "MySQL (PDO)" "MySQLi" "MySQL (PDO)" "PostgreSQL (PDO)")
hosts=("unix:/var/run/mysql-socket/mysqld.sock" "unix:/var/run/mysql-socket/mysqld.sock" "unix:/var/run/mariadb-socket/mysqld.sock" "unix:/var/run/mariadb-socket/mysqld.sock" "unix:/var/run/postgresql-socket")
ports=("" "" "" "" "")

Failed for mysqli and mariadbi with 500 server error with:

Joomla\Database\DatabaseDriver::extractHostPortSocket(): Argument #1 ($host) must be of type string, null given, called in /var/www/html/libraries/vendor/joomla/database/src/Mysqli/MysqliDriver.php on line 203

@richard67
Copy link
Contributor Author

Hmm the MySQLi and the PDO driver set the host to „localhost“ in the constructor, so a null host value should never happen.

@richard67
Copy link
Contributor Author

Will check later why that fails.

@muhme
Copy link

muhme commented Oct 29, 2024

To verify, I created a new installation using the database-315 patch instead, ran test No.9 Unix sockets and it is successfully passed.

@richard67
Copy link
Contributor Author

Yes, I know what the problem is. The method can run multiple times when connecting for checking the database and then disconnecting and connecting again to make the installation.

In the old code before PR #310 the $this->options['host'] set to null at the first run and then was just passed again to the preg_replace at the 2nd run, which causes a deprecation notice on newer PHP version but not an error.

Now we have a function where the parameter is not nullable.

There are 2 ways to solve it: Either I do it like for PDO and do not modify the $this->options but use local variables, so we have the same starting conditions with every new connect. Or I make the $hostparameter nullable and do an early return of the unmodified parameters when that parameter is null.

I tend to the first way.

The real clean way would be to use the method not in the connect method but in the constructor so it would run only one time on a particular instance.

@richard67
Copy link
Contributor Author

Hmm, on the other hand, the 2nd way would make it behave on MySQL like it was before PR #310 .

@richard67
Copy link
Contributor Author

9. 🟥 Test Unix sockets by adapting `test.sh` and running it again:
types=("MySQLi" "MySQL (PDO)" "MySQLi" "MySQL (PDO)" "PostgreSQL (PDO)")
hosts=("unix:/var/run/mysql-socket/mysqld.sock" "unix:/var/run/mysql-socket/mysqld.sock" "unix:/var/run/mariadb-socket/mysqld.sock" "unix:/var/run/mariadb-socket/mysqld.sock" "unix:/var/run/postgresql-socket")
ports=("" "" "" "" "")

Failed for mysqli and mariadbi with 500 server error with:

Joomla\Database\DatabaseDriver::extractHostPortSocket(): Argument #1 ($host) must be of type string, null given, called in /var/www/html/libraries/vendor/joomla/database/src/Mysqli/MysqliDriver.php on line 203

@muhme Fixed. Thanks for testing so far.

@muhme
Copy link

muhme commented Oct 30, 2024

✅ Successfully completed the test with all 9 steps again, confirming that all tests passed smoothly this time. 👍

@HLeithner
Copy link
Contributor

I have some questions.

  1. It seems hard to validated a ipv4 or ipv6 address with all it's implications, maybe using a lib for it is a better approach or at least look at it and make own code better, my first hit was https://github.com/mlocati/ip-lib no idea if it's good but it also support ipv6 zones
  2. What's the reason todo the detection on connect and not in the constructor? If I saw it right we have no function to change the options later, and if we have we might simple run an HostPortWhateverupdate function with the new DSN string?

wouldn't this reduce a bit of complexity? at least I read somewhere something about double decode issue.

@richard67
Copy link
Contributor Author

@HLeithner Regarding your question 2: I don’t know why it is done in the connect method. It was like that since long time, if not since ever, when I checked the history on the 1.0-dev branch. I did not want to change that in a patch version. I fully agree that it should be done in the constructor. If there is consensus about that I can change this PR here (or make a new one).

Regarding question 1: Same reason as question 2, I did not want to make such a big change now.

We could discuss that in the next maintainers meeting for framework topics.

@richard67
Copy link
Contributor Author

richard67 commented Nov 9, 2024

I've created an alternative PR which makes the same changes as this one here (except of adding a unit test) plus moves the modification of options from the connect method to the constructor.

Closing in favour of #317 . Thanks for feedback and tests.

@richard67 richard67 closed this Nov 9, 2024
@richard67 richard67 deleted the 3.x-dev-ipv6-square-brackets branch November 10, 2024 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants